The use of portable electronic devices and mobile communication devices has increased dramatically in recent years. Moreover, the demand for video enabled mobile devices is rapidly increasing. Video processing requires a significant amount of signal processing and places high processing demand on a mobile device having limited computational power and battery power. Accordingly, video is generally received in a compressed format to reduce the amount of data required to represent the images. The compressed data also facilitates real time data delivery as the amount of data to be transmitted is decreased. Video coding is the process of encoding the video into a compressed format.
In traditional video compression, a video sequence is encoded into a compressed bitstream, which can later be decoded to obtain a reconstruction of the original video sequence. This system consists of one encoder and one decoder. Video compression can be extended upon to provide scalability. Scalability allows for an adjustment of video quality based on available hardware or software resources. Scalability also provides a platform for seamless mobility and which allows users to efficiently consume video contents across different video devices and transmission channels. For example, certain hardware may only support a range of framerates or a range of bitrates. A scalable video compression system allows the hardware to support various decoding options scaling to the hardware resources.
Scalable video compression (SVC) produces an embedded bitstream that can be truncated at different segmentation points (i.e. locations within the embedded bitstream) to produce reconstructed videos with different desired parameters such as resolution, framerate, and quality. As an example, referring to FIG. 1, a scalable video compression (SVC) system can consist of an encoder 120, a bitstream extractor 140, and a decoder 160. The encoder 120 can compress the video input and generate an embedded bitstream 130 with bits representing the video input. The SVC system 100 can receive user input parameters 145 for setting a video decoding quality. The SVC system 100 is scalable in the sense that the video decode quality can be scaled in accordance with the parameters 145.
Referring to FIG. 2, an exemplary scalable bitstream representation is shown. It should be noted that the bitstream segments (B0-B35) can be saved in various orderings. In the illustration of FIG. 2, the bits are shown in a row (e.g. temporal layers) and column (e.g. spatial levels) format. In practice, each bit 131 of the bitstream is received in a sequential manner that may be dictated by the encoding order. The progressive refinement (PR) slices in the refinement layers of a scalable bitstream can be truncated at any location in the related bitstream segments, with each additional bit providing an improvement to reconstructed video quality. For example, the scalable bitstream may be truncated at the end of the bitstream segment B14 thereby providing only bit segments B0 through B14 for decoding which may be sufficient for a low quality image. Referring back to FIG. 1, the bitstream extractor 140 can truncate an encoded bitstream to obtain a stream at a lower bit rate that will decode at a desired spatial resolution and temporal frame rate, with a SNR quality that is based on the provided user input parameters 145. A range of bit-rates is allowed at each resolution and framerate, with higher rates yielding better SNR quality. For constant bitrate video coding applications, the bitstream extractor shall sequentially collect the relevant bit segments from the given scalable bitstream following the designed bitstream extraction path until the target bit budget is reached. While a theoretically optimal extraction ordering is not clear, various schemes of bit extraction can perform differently for a wide variety of video sequences.
There are three main types of scalability: spatial, temporal, and SNR (quality). Spatial scalability is the ability to decode the bitstream at many different resolutions. Temporal scalability is the ability to decode the bitstream at many different frame rates. SNR scalability is the ability to decode the bitstream at different bit rates to achieve desired quality. For example, a user can adjust parameters 145, such as resolution, frame rate, and quality for setting the spatial, temporal, and SNR scalability. Spatial and temporal scalability are generally provided in a layered manner, where there are a small set of possible resolutions and frame rates. That is, information is provided in layers such that each layer provides an incremental improvement in the video decoding quality. It is possible to provide SNR scalability in a similar layered manner, or in a more continuous manner with an embedded bitstream.